Plugin Generator

Plugins allow you to provide your own generator.

  • You can write in any language
  • Simple: a plugin just responds to RPC HTTP requests.
  • You can use it in a sidecar, or standalone deployment.
  • You can get your plugin running today, no need to wait 3-5 months for review, approval, merge and an Argo software release.
  • You can combine it with Matrix or Merge.

To start working on your own plugin, you can generate a new repository based on the example applicationset-hello-plugin.

Simple example

Using a generator plugin without combining it with Matrix or Merge.

  1. apiVersion: argoproj.io/v1alpha1
  2. kind: ApplicationSet
  3. metadata:
  4.   name: myplugin
  5. spec:
  6.   generators:
  7.     - plugin:
  8.         # Specify the configMap where the plugin configuration is located.
  9.         configMapRef:
  10.           name: my-plugin
  11.         # You can pass arbitrary parameters to the plugin. `input.parameters` is a map, but values may be any type.
  12.         # These parameters will also be available on the generator's output under the `generator.input.parameters` key.
  13.         input:
  14.           parameters:
  15.             key1: "value1"
  16.             key2: "value2"
  17.             list: ["list", "of", "values"]
  18.             boolean: true
  19.             map:
  20.               key1: "value1"
  21.               key2: "value2"
  22.               key3: "value3"
  24.         # You can also attach arbitrary values to the generator's output under the `values` key. These values will be
  25.         # available in templates under the `values` key.
  26.         values:
  27.           value1: something
  29.         # When using a Plugin generator, the ApplicationSet controller polls every `requeueAfterSeconds` interval (defaulting to every 30 minutes) to detect changes.
  30.         requeueAfterSeconds: 30
  31.   template:
  32.     metadata:
  33.       name: myplugin
  34.       annotations:
  35.         example.from.input.parameters: "{{ generator.input.parameters.map.key1 }}"
  36.         example.from.values: "{{ values.value1 }}"
  37.         # The plugin determines what else it produces.
  38.         example.from.plugin.output: "{{ something.from.the.plugin }}"
  • configMapRef.name: A ConfigMap name containing the plugin configuration to use for RPC call.
  • input.parameters: Input parameters included in the RPC call to the plugin. (Optional)

Note

The concept of the plugin should not undermine the spirit of GitOps by externalizing data outside of Git. The goal is to be complementary in specific contexts. For example, when using one of the PullRequest generators, it’s impossible to retrieve parameters related to the CI (only the commit hash is available), which limits the possibilities. By using a plugin, it’s possible to retrieve the necessary parameters from a separate data source and use them to extend the functionality of the generator.

Add a ConfigMap to configure the access of the plugin

  1. apiVersion: v1
  2. kind: ConfigMap
  3. metadata:
  4.   name: my-plugin
  5.   namespace: argocd
  6. data:
  7.   token: "$plugin.myplugin.token" # Alternatively $<some_K8S_secret>:plugin.myplugin.token
  8.   baseUrl: "http://myplugin.plugin-ns.svc.cluster.local."
  • token: Pre-shared token used to authenticate HTTP request (points to the right key you created in the argocd-secret Secret)
  • baseUrl: BaseUrl of the k8s service exposing your plugin in the cluster.

Store credentials

  1. apiVersion: v1
  2. kind: Secret
  3. metadata:
  4.   name: argocd-secret
  5.   namespace: argocd
  6.   labels:
  7.     app.kubernetes.io/name: argocd-secret
  8.     app.kubernetes.io/part-of: argocd
  9. type: Opaque
  10. data:
  11.   # ...
  12.   # The secret value must be base64 encoded **once**
  13.   # this value corresponds to: `printf "strong-password" | base64`
  14.   plugin.myplugin.token: "c3Ryb25nLXBhc3N3b3Jk"
  15.   # ...

Alternative

If you want to store sensitive data in another Kubernetes Secret, instead of argocd-secret, ArgoCD knows how to check the keys under data in your Kubernetes Secret for a corresponding key whenever a value in a configmap starts with $, then your Kubernetes Secret name and : (colon) followed by the key name.

Syntax: $<k8s_secret_name>:<a_key_in_that_k8s_secret>

NOTE: Secret must have label app.kubernetes.io/part-of: argocd

Example

another-secret:

  1. apiVersion: v1
  2. kind: Secret
  3. metadata:
  4.   name: another-secret
  5.   namespace: argocd
  6.   labels:
  7.     app.kubernetes.io/part-of: argocd
  8. type: Opaque
  9. data:
  10.   # ...
  11.   # Store client secret like below.
  12.   # Ensure the secret is base64 encoded
  13.   plugin.myplugin.token: <client-secret-base64-encoded>
  14.   # ...

HTTP server

A Simple Python Plugin

You can deploy it either as a sidecar or as a standalone deployment (the latter is recommended).

In the example, the token is stored in a file at this location : /var/run/argo/token

  1. string-password
  1. import json
  2. from http.server import BaseHTTPRequestHandler, HTTPServer
  4. with open("/var/run/argo/token") as f:
  5.     plugin_token = f.read().strip()
  8. class Plugin(BaseHTTPRequestHandler):
  10.     def args(self):
  11.         return json.loads(self.rfile.read(int(self.headers.get('Content-Length'))))
  13.     def reply(self, reply):
  14.         self.send_response(200)
  15.         self.end_headers()
  16.         self.wfile.write(json.dumps(reply).encode("UTF-8"))
  18.     def forbidden(self):
  19.         self.send_response(403)
  20.         self.end_headers()
  22.     def unsupported(self):
  23.         self.send_response(404)
  24.         self.end_headers()
  26.     def do_POST(self):
  27.         if self.headers.get("Authorization") != "Bearer " + plugin_token:
  28.             self.forbidden()
  30.         if self.path == '/api/v1/getparams.execute':
  31.             args = self.args()
  32.             self.reply({
  33.                 "output": {
  34.                     "parameters": [
  35.                         {
  36.                             "key1": "val1",
  37.                             "key2": "val2"
  38.                         },
  39.                         {
  40.                             "key1": "val2",
  41.                             "key2": "val2"
  42.                         }
  43.                     ]
  44.                 }
  45.             })
  46.         else:
  47.             self.unsupported()
  50. if __name__ == '__main__':
  51.     httpd = HTTPServer(('', 4355), Plugin)
  52.     httpd.serve_forever()

Execute getparams with curl :

  1. curl http://localhost:4355/api/v1/getparams.execute -H "Authorization: Bearer string-password" -d \
  2. '{
  3.   "applicationSetName": "fake-appset",
  4.   "input": {
  5.     "parameters": {
  6.       "param1": "value1"
  7.     }
  8.   }
  9. }'

Some things to note here:

  • You only need to implement the calls /api/v1/getparams.execute
  • You should check that the Authorization header contains the same bearer value as /var/run/argo/token. Return 403 if not
  • The input parameters are included in the request body and can be accessed using the input.parameters variable.
  • The output must always be a list of object maps nested under the output.parameters key in a map.
  • generator.input.parameters and values are reserved keys. If present in the plugin output, these keys will be overwritten by the contents of the input.parameters and values keys in the ApplicationSet’s plugin generator spec.

With matrix and pull request example

In the following example, the plugin implementation is returning a set of image digests for the given branch. The returned list contains only one item corresponding to the latest built image for the branch.

  1. apiVersion: argoproj.io/v1alpha1
  2. kind: ApplicationSet
  3. metadata:
  4.   name: fb-matrix
  5. spec:
  6.   goTemplate: true
  7.   generators:
  8.     - matrix:
  9.         generators:
  10.           - pullRequest:
  11.               github: ...
  12.               requeueAfterSeconds: 30
  13.           - plugin:
  14.               configMapRef:
  15.                 name: cm-plugin
  16.               input:
  17.                 parameters:
  18.                   branch: "{{.branch}}" # provided by generator pull request
  19.               values:
  20.                 branchLink: "https://git.example.com/org/repo/tree/{{.branch}}"
  21.   template:
  22.     metadata:
  23.       name: "fb-matrix-{{.branch}}"
  24.     spec:
  25.       source:
  26.         repoURL: "https://github.com/myorg/myrepo.git"
  27.         targetRevision: "HEAD"
  28.         path: charts/my-chart
  29.         helm:
  30.           releaseName: fb-matrix-{{.branch}}
  31.           valueFiles:
  32.             - values.yaml
  33.           values: |
  34.             front:
  35.               image: myregistry:{{.branch}}@{{ .digestFront }} # digestFront is generated by the plugin
  36.             back:
  37.               image: myregistry:{{.branch}}@{{ .digestBack }} # digestBack is generated by the plugin
  38.       project: default
  39.       syncPolicy:
  40.         automated:
  41.           prune: true
  42.           selfHeal: true
  43.         syncOptions:
  44.           - CreateNamespace=true
  45.       destination:
  46.         server: https://kubernetes.default.svc
  47.         namespace: "{{.branch}}"
  48.       info:
  49.         - name: Link to the Application's branch
  50.           value: "{{values.branchLink}}"

To illustrate :

  • The generator pullRequest would return, for example, 2 branches: feature-branch-1 and feature-branch-2.

  • The generator plugin would then perform 2 requests as follows :

  1. curl http://localhost:4355/api/v1/getparams.execute -H "Authorization: Bearer string-password" -d \
  2. '{
  3.   "applicationSetName": "fb-matrix",
  4.   "input": {
  5.     "parameters": {
  6.       "branch": "feature-branch-1"
  7.     }
  8.   }
  9. }'

Then,

  1. curl http://localhost:4355/api/v1/getparams.execute -H "Authorization: Bearer string-password" -d \
  2. '{
  3.   "applicationSetName": "fb-matrix",
  4.   "input": {
  5.     "parameters": {
  6.       "branch": "feature-branch-2"
  7.     }
  8.   }
  9. }'

For each call, it would return a unique result such as :

  1. {
  2.   "output": {
  3.     "parameters": [
  4.       {
  5.         "digestFront": "sha256:a3f18c17771cc1051b790b453a0217b585723b37f14b413ad7c5b12d4534d411",
  6.         "digestBack": "sha256:4411417d614d5b1b479933b7420079671facd434fd42db196dc1f4cc55ba13ce"
  7.       }
  8.     ]
  9.   }
  10. }

Then,

  1. {
  2.   "output": {
  3.     "parameters": [
  4.       {
  5.         "digestFront": "sha256:7c20b927946805124f67a0cb8848a8fb1344d16b4d0425d63aaa3f2427c20497",
  6.         "digestBack": "sha256:e55e7e40700bbab9e542aba56c593cb87d680cefdfba3dd2ab9cfcb27ec384c2"
  7.       }
  8.     ]
  9.   }
  10. }

In this example, by combining the two, you ensure that one or more pull requests are available and that the generated tag has been properly generated. This wouldn’t have been possible with just a commit hash because a hash alone does not certify the success of the build.